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Abstract 

The crystal structure determines both the Fermi surface and pairing symmetry of the supercon- 
ducting metals. It is demonstrated in the framework of the general phenomenological approach that 
this is of the primary importance for the determination of the structure of the Fulde-Ferrell-Larkin- 
Ovchinnikov (FFLO) phase in the magnetic field. The FFLO modulation of the superconducting 
order parameter may be revealed in the form of the higher Landau level states or/and modula- 
tion along the magnetic field. The transition between different FFLO states could occur with the 
temperature variation or with the magnetic field rotation. 
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I. INTRODUCTION 



It is well known that in type II superconductors the Abrikosov vortex state can be formed 
under a magnetic field. In most cases the destruction of superconductivity happens due to 
the orbital effect. However there can be a situation when paramagnetic effect plays an 
important role in destruction of superconductivity (magnetic field acting only on electron 
spins). In this case the non-uniform Fulde-Ferrell-Larkin-Ovchinnikov (FFLO) state [1, 2] 
appears in superconductors, which is characterized by the modulation of the order parameter. 
The structure of the FFLO phase in the real compounds may be very rich [3-12]. Interplay 
of orbital and paramagnetic effects has been described in the isotropic model by Gruenberg 
and Gunther in [3], where they calculated critical field and structure of the order parameter. 
It was found in [3] that the orbital effect is detrimental to the FFLO state, but still such a 
state can exist if the ratio of pure orbital effect iJS'^(O) and pure paramagnetic limit Hp{0) 
is larger than 1.28, i.e. the Maki parameter aM = V^H"2^(0) / Hp(0) must be larger 1.8. 
Pure paramagnetic hmit at T = can be estimated as Hp{0) — Ao/a/2//b, where Aq is 
BCS gap at T = and //b is the Bohr magneton. In [3] the modulation was studied using 
a zero Landau level function, which holds true only for moderate Maki parameter ccm < 9. 
It was found in [4] that for large values of Maki parameter > 9 the higher Landau level 
solutions become relevant. In this case the critical field Hc2{T) consists of several curves 
each corresponding to a different Landau level solution. The analysis of the orbital effect in 
the FFLO state [3, 4] were performed for the isotropic metals with s-wave type of pairing. 
However in [13] it was demonstrated that it readily generalized for the case of the metals 
with elliptic Fermi surface. In such a case the Maki parameter becomes angular dependent. 

For example for the case of the quasi-2D or anisotropic 3D superconductors au increases 
dramatically for the in-plane field orientation. Therefore we may expect the transitions 
between the usual FFLO state with zero Landau levels [3] to the state with higher Landau 
levels [4-7] when the magnetic field is tilted from the perpendicular orientation to the parallel 
one. Also crossover from the pure FFLO state to the vortex states with higher Landau levels 
indexes in the model of quasi-2D system has been predicted in [14] . In real compounds the 
deviation of the Fermi surface from the elliptic form is crucial for the adequate description 
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of the FFLO state as well as the type of the superconductivity pairing (e.g. s- or d-wecve) 
[15-17]. This circumstance is related with a fact that the description of the FFLO state 
in the framework of Ginzburg-Landau approach needs the consideration of the higher-order 
derivatives of the order parameter in addition to the usual gradient terms. For example, 
in the case of pure paramagnetic effect the critical field and modulation vector q strongly 
depend on anisotropy or nesting properties of the Fermi surface [18-21]. In this article we 
consider a realistic case with a non-elliptic Fermi surface and for the definiteness we restrict 
ourself to the tetragonal symmetry. For example quasi-two-dimensional superconductor 
CeCoIns [9] provides favorable conditions for the formation of the FFLO state and it has a 
tetragonal symmetry. 

A characteristic feature of the FFLO state is the existence of a tricritical point (TCP) 
in the field-temperature phase diagram [22]. TCP is the meeting point of three transition 
lines separating the normal metal, the uniform superconductivity and the FFLO state. 
Formation of the FFLO state near the TCP may be described by modified Ginzburg-Landau 
functional (MGL) [23]. Appearance of the non-uniform state is related with a change of the 
sign of the coefficient g at the gradient term g \ in the free energy density. In the 

standard Ginzburg-Landau theory the coefficient g is positive. Here it vanishes at the TCP 
(T*, Hc2{T*)), and then becomes negative for T <T*. The absolute value of g grows as we 
move further from the TCP, for example with increasing of the magnetic field or lowering 
temperature. A negative g means that the modulated state has a lower free energy than the 
uniform one. In order to obtain the modulation vector one needs to include the term with 
higher order derivatives in the MGL functional [23]. 

In this paper we study the effects of crystal (or pairing) anisotropy on the FFLO phase. 
Using MGL approach we introduce free energy density describing tetragonal system. We 
examine the case of the Fermi surface close to elliptic one. Therefore JF can be divided into 
isotropic and perturbative parts. We demonstrate that the higher Landau level solutions 
may be realized for arbitrary values of Maki parameter in contrast with isotropic model. This 
is a special mechanism of the higher Landau level phase formation in 3D system. Moreover 
depending on various type of deviation of the Fermi surface from isotropic form three possible 
solutions for the FFLO state can be realized: (a) maximum modulation occurs along the 
magnetic field with zero Landau level state, (b) both modulation and higher Landau level 
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state, (c) highest possible Landau level and no modulation along the field (or modulation 
with very small wave- vector) . Moreover due to the specific form of the Fermi surface the 
variation of magnetic field orientation may provoke transitions between the states with 
different Landau levels. 

The main goal of the present paper is to demonstrate that in the presence of the orbital 
effect and for the realistic Fermi surface the very different types of the FFLO state could 
be realized. In particular, if the preferred modulation direction is perpendicular to the 
magnetic field this can results in the formation of the higher Landau levels mixed state 
with no modulation along the field at all. Our approach is fully justified near the TCP 
and for superconductors with large Maki parameters. However qualitatively it provides 
the understanding of the FFLO state at all temperatures and for arbitrary strength of the 
orbital effect. Here we calculate the line of the second order transition from the normal to 
the superconducting state. For this purpose we use the quadratic over the superconducting 
order parameter MGL functional. To describe the properties of the FFLO state it is needed 
to retain the higher order terms over the superconducting order parameter. The situation is 
completely analogous with that of the Abrikosov vortex lattice. From the symmetry reasons 
it is clear that the transitions between the mixed states describing by the different Landau 
levels will be the first order transitions. However the appearance of the modulation along the 
magnetic field may occur through a continuous transition. All these interesting questions 
deserve further studies but they are well beyond the scope of the present article. 



II. FFLO STATE IN ANISOTROPIC GINZBURG-LANDAU MODEL 

The most general form of the MGL functional quadratic over ^ is 

3 3 
1=1 i=l i^j 

where a{H^ T) = ao{T — Tcu{H)), Tcu{H) is transition temperature into the uniform super- 
conducting state, Hj = —ih-^ — are momentum operators and Ai are the components 
of the vector potential, further we put h = 1. Assuming the tetragonal symmetry in the 
case of the pure elliptic Fermi surface we have following relation for the effective mass 
rriz 7^ rrix — my — m. The elliptical Fermi surface could be transformed into isotropic one 
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by the following scaling transformation z' — ^/rn^Jmz [13]. Components of the magnetic 
field also transform as H — > H' — {^Hx, ^Hy, Hz). Further we suppose that actual 
Fermi surface deviation from the elliptical form is small and after corresponding scahng 
transformation the functional (1) for the tetragonal symmetry is written as 



=1 

e 



i=l 



Ez In^^l' + ^ (|n,n,^|' + |n,n,^|') (2) 



+- (|n,n,^r + in.n.^r + |n.n,^|' + |n,n,^|') . 

Coefficients 51, 7, s^, e depend on structure of the Fermi surface, but for the FFLO appear- 
ance g must be positive. The terms —gY^i=\ |ni^|^ + 7 |X]i=i n^^|^ describe the elliptic 
form of the Fermi surface (for s-wave superconductivity) and terms with coefficients 
and e are considered as perturbation. 

Without orbital effect the momentum operators are simplified to Hj — the 
solution for the order parameter could be presented as ^ = ^gexp(i;g ■ r). In the case 
of elliptic Fermi surface we have a degeneracy over direction of the FFLO modulation q. 
The crystal structure effects are expressed via the terms with s^, e^^ s and they lift this 
degeneracy and determine the direction of the FFLO modulation. The free energy density 
is written as 

= |q! — gcf + 7^"^ + e^q^ cos* 9 + e^q* sm* 9 cos^ (p sin^ (p + eq'* sin^ 9 cos^ 6'} l^'gl^ , 

(3) 

where we have used spherical system for q: q^ — qcosipsva.9, qy — qsvcnpsva.9, qz — qcos9. 

Due to the tetragonal symmetry the modulation in xy-plane is either parallel to x or y axis 
{Ex > 0) or along the bisector (e^ < 0). For definiteness we may suppose that > 0. Note 
that in the case < the rotation of the xy axis by 7r/4 provides us the same functional 
(3) with renormalized coefficients e'^ and e' but with Ex > 0. The wave vector of modulation 
q will be in xy-plane if e > , < 0, parallel to z axis if £ > 0, > (see Fig. 1). For 
the region e < 0, £ < 28^ the direction of modulation is at angle 9 — ^ arccos to the 
xy-plane. Therefore the crystal anisotropy (or/and pairing anisotropy) lifts the degeneracy 
over the direction of the FFLO modulation and the whole diagram in the (s^, e) plane is 
presented in Fig. 1. 




FIG. 1: Modulation (£,£2) diagram in the case of the absence of the orbital effect (pure param- 
agnetic limit). Areas with different patterns correspond to different orientation of the wave- vector 
modulation. The phase diagram does not depend on the value. 

III. ORBITAL EFFECT FOR THE MAGNETIC FIELD APPLIED ALONG Z 
AXIS 

The exact solution of the linearized equation for \l/(r) is unavailable in general case if the 
orbital effect is taken into account. The Landau level solution with additional modulation 
along the field works only for the case of eUiptical Fermi surface. In this case we obtain the 
degeneracy over modulation q and Landau level n, when we move from the TCP. However if 
the anisotropy effects are taken into account they lift this degeneracy. We demonstrate that 
depending on the parameters of the system very different types of the FFLO state could be 
realized. 

We begin with the case when magnetic field H is applied along tetragonal z axis and 
the gauge is chosen as yl=(7/if, 0, 0). Following for example Ref. [7] we can express our 
operators Ilj using boson operators of creation r] and annihilation 7^+ as 11^: = (77 — 77"*"), 

Ily = -^^^ (77 + 77"*"), where — -sp^- Our goal is to find Tc{H), which is the transition 
temperature into the FFLO state, i.e. we need to find the solution which gives the maximum 
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of a{H,T) — ao{Tc — Tcu{H)). To do this we use the variation method [24] and look for a 
maximum of a written as 





1*1 









a{H,T) = max< ^ ' 2 }■ (4) 



In general case ^{x,y,z) — YliCn^n{x,y)e'^'^'^ , but since the anisotropy effects are small 
we can approximate our solution only with a single Landau level function (pi^ [25], which 
is well known solution for the system with isotropic form of the Fermi surface. In our 
calculations we use the following properties of Landau functions: / ipn'{>md^f — ^nm, Vfn — 
\fn(fn-i, iT^fn — Vn-\-l(pn+i', wc also normalize C„ so that J {"^fd^r = 1. To neglect the 
other Landau level functions {n ^ N) in our representation their corresponding coefficients 
should be small comparing to the coefficient Cn. This leads to the following condition for 

the applicability of the single level approximation ^Jj^ ^ ^ which is determined by the 
Fermi surface deviation e from the elliptic form {e is of the order of average between e, 
and and its exact value depends on magnetic field orientation). Calculating a (if, T) using 
the operators r] and rf" we can express it in terms of and = ^^^(2n + 1) as 



a[H, T) = max {g [ql + C'] " 7 [ql + C'] ' (5) 

4 j^— 4 ~ 2/-— 2 ^ I 

If we consider only unperturbed part g [q^ + ^~^] —7 [g^ + ^~^]^, put u = q^+^n"^ and take 
derivative with respect to u we obtain the maximum point at Uq = g/2'~f. When the system is 
close to the TCP, g is small so that g/2j < In this case we obtain the lowest Landau level 
n — Q and no modulation along 2; (g'^ = 0). If we move further from the TCP and g grows 
(C^^ < g/2^ < 3^^^) then n remains equal to 0, but is not. If Maki parameter is large then 
our approach will still be valid even far away from the TCP. In this case g/2'^ > "i^J^ and n 
can be larger than 0. Hence, we have degeneracy over choosing of Landau level n and g^. But 
as was written earlier taking into account small perturbative terms with e:^, e^,, e we remove 
this degeneracy and find the maximum a{H,T) with a respect to q^ — ^/uq — There 
are three possible types of solutions (combinations of n and q^) depending on ez,Sx,s: (a) 
maximum modulation q^ — ^ g/2j — and zero Landau level n; (b) non zero modulation 
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^ — 4s) / (SEz + Ex — Se) with n = [I {Ch {uq — qi) — 1)] ,where brackets [ ] mean 
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4^^H 2 



and near 



that only integer part is taken; (c) highest possible Landau level n - 
zero modulation. All these cases are shown in Fig. 2. However due to integer nature of 
n the modulation has a very special behavior in cases b) and c), it changes abruptly 
every time when our solution jumps from one to another Landau level. It should be noted 
that in the case c) the wave- vector of modulation instead of being zero oscillates with H 
(or with g, when we move further form the TCP) due to the mismatch of ito = g/'^J and 
= ^]f{2n + 1). The diagram shown in Fig. 2 looks similar to that in Fig. 1 with the 
exception of the shift along both axis due to the presence of the Ex coefficient. When we get 
zero wave-vector of modulation in Fig. 2 the same area in Fig. 1 corresponds to modulation 
in xy-plane. If modulation vector q along the applied magnetic field is zero then modulation 
could arise in the direction perpendicular to the field. When n and qz is intermediate (not 
maximum and not zero, lower right part of the diagrams) FFLO modulation could be formed 
in both perpendicular and parallel directions to the field (g^-l-gi = — It should be 
noted that we cannot achieve a smooth transition from one diagram to another by decreasing 

to due the condition of single Landau level approximation ^ fy^' despite the 
fact that the only difference between diagrams in Figs. 1 and 2 is shift of the intersection 
point due to the presence of the Ex coefficient. The same is true for the case H \\ x where 
difference will be more significant. 

Mciximum Landau level and residual modulation. Due to the fact that n is integer 

the wave-vector of modulation q^ may not be equal exactly to zero but to some value 



less than a/mq — ^n'^ when n is maximum. To calculate this value we maximize a{H,T) 
from the Eq. (5) again but this time ^^7^ = ^J^{2n + 1) will be treated as constant. We 
obtain q1 = max ^0, j and the general solution for residual modulation q^ 

will oscillate with H or with g (absolute value of g is increasing when we are moving away 
from the TCP). Wave vector of modulation is zero when is close to uq — g/2^, then 
is start to increase linearly with g until it drops to zero again when the solution "jumps" 
to another Landau level (Fig. 3). With the increasing effect of anisotropy, the area of zero 
wave-vector modulation widens, and in the limiting case it will cover all parameter range. 
Similar results were obtained for isotropic case at au > 9 at low temperature [4]. 



8 




FIG. 2: Modulation diagram in the case when the magnetic field applied along z axis. There are 3 
areas on the diagram corresponding to 3 types of the solution for modulation vector and Landau 
level n. Modulation direction is always parallel to the applied field and here is chosen equal to 
7- 




FIG. 3: Dependence of residual modulation on the parameter g (normalized to ^^J^)- The wave 
vector of modulation q is shown here by the solid line and measured in units of . The values 
of Landau level n is shown by the dashed line. The parameter e is chosen here as O.27. 
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IV. THE CASE OF THE MAGNETIC FIELD APPLIED ALONG X AXIS 



For magnetic field applied along the x axis we choose A as (0, 0, yif), the order parameter 
^(x, y,^) = <^iv(z/)e*^^^ where qx is modulation along the x axis, and reintroduce creation 
and annihilation operators as 77 = ^ {Jly — iHz) and — ^ ij^y + ^n^). Repeating the 
same calculations as for the H\\z case we have 



a{H, T) = max [g [ql + C'] - 7 K + C'] ' (6) 

Again we put u = q^. + and find that maximum point Uq = g/2'~f is the same for 
unperturbed part. The degeneracies over q^ and n are removed in a similar way by taking 
into account the perturbative terms e^, s^, s. Diagram for maximums of a{H, T) in the case 
with magnetic field applied along x axis is shown in Fig. 4. Three main areas of the diagrams 
are similar to the ones shown in Fig. 2: (a) maximum modulation q^ — ^ — ^Jj^ and zero 
Landau level n; (b) non zero modulation q^ = {SSz — e — 2Sx) / (Ss^ — 3s — Asx) with 
n = [I (^If (mo — q^) — 1)]; (c) highest possible Landau level n = -§^Ch ~ \ and residual 
modulation described earlier. Diagrams shown in Figs. 4 and 1 have some similarities with 
a respect to the conditions for the different types of solutions. On both diagrams the FFLO 
modulation along x (or y) axis corresponds to the upper-right quarter and there is no 
modulation in xy plane in the left quarters of the diagrams. 



V. MAGNETIC FIELD APPLIED IN XY PLANE 



If a magnetic field H is applied in the a;|/-planc {(3 is the angle between H and x axis), 
then it is convenient to rotate x, y axis around z by angle (3 to reduce the problem to the 
case H || x. Under this rotation the terms with coefficients g, 7, Sz remain unchanged and 
the rest parts are transformed according to rules x' — x cos /3 + y sin /3, y' — y cos (5 — xsva.(5 
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FIG. 4: Modulation diagram (£,£2) in the case when the magnetic field is applied along x axis. 
There are three areas on the diagram corresponding to different types of the solution for modulation 
vector Qx and Landau level n. Modulation direction is always parallel to the applied field. The 
position of the intersection point is determined by the coefficient Sx- 



Tlx = n;cos/?-n;sin/? (7) 

Yly = n^sin/3 + n^cos/3 (8) 

n. = K = -i^ - -y'H (9) 

oz c 

The operators 77 and 77+ are expressed using new W^, and as before in the case 
H ^ X. Due to the symmetry of the problem only Sx term in Eq. (2) acquires the dependence 
on (5 in the final expression for a{H,T) 



a 



{H, T) = max [g [<^^ + C^] - 7 [q'^ + C^] ' 



-e.lCn' - \ex4lC' - U - H'C' + + Uh ) (10) 



2^^x^n g^" '^Zg'^H 

where q'x is modulation along the new x' axis parallel to the magnetic field H. Directly from 
the Ex term it can be concluded that the angles /3 = 0, ±|, tt will lead to the old results, when 
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magnetic field is applied along x or y axis. The main results for the maximum of a{H, T) 
will be similar to the case H \\ x, with the only exception that separation lines on phase 
diagram are changed to three fines: e — —Se^ + | sin^ ^Pe^, e = Ss^ + ^i^^ 2/3 — 2) and 
£ = (I sin^ 2/3 — l) Ex- However this change only affects the initial shift of the diagram from 
the center. For example in Fig. 5 the case (3 = |(n+|) is shown and the intersection point has 
shifted to the opposite quarter of the graph. For general values of the intersection point is 

situated at {ez, e) = ((— ^ sin^ 2/5 + |) Ex, (| sin^ 2/5 — l) Ex) on a line segment connecting 
the two intersection points for (5 — and /? — i). On phase diagram like in previous 

cases we have 3 possible solutions: 

1. q'^ = Ho (3£^ - £ - 2£^ + f sin^ 2/3 £^) / (3£^ - 3£ - 4£^ + f sin^ 2/3 £^) , n = 
[l i^H (^^0 - q'x) - 1)] , when £ < min (3£;, - 2£^ + f sin^ 2(3e^, £^(| sin^ 2(3e^ - 1)) . 

2- 9x ~ (near zero residual modulation), maximum n in this case will be equal to 
[m^h ~ I] ■ This corresponds to -3£2 + | sin^ 2(3e^ > e > Se^ - 2e^ + ^ sin^ 2/3£^. 

3. q'^ = ■\Jg/2'y — - maximum modulation along the magnetic field with n = 0. This 
corresponds to £ > max {ex{\ sin^ 2j3Ex — 1), — 3£2 + | sin^ 2j3ex) ■ 

Using q'j. corresponding to the maximum oia{H, T) we find Tc{H). When the parameters 
of our system (the actual values of e^SziSx coefficients) correspond to the point in (£, £^) 
plane situated near one of the separation lines in Figs. 4 or 5 then the magnetic field rotation 
can lead to transition between the corresponding two phases. The simplest case with £ = 0, 

£2; = is shown in the Figs. 6 and 7. For positive £^ the transition is between states 
{q — Q,n — max) and (g > 0, n > 0); for negative Sx the two states are {q — max, n — Q) and 
(q' > 0, n > 0). Integer nature of Landau level n manifests itself in the state (g > 0, n > 0), 
when the FFLO modulation could change several Landau levels while magnetic field rotates 
in a given region. In this case Tc line consists of several curves each corresponding to a 
different Landau level solution. If switching between the different solutions does not occur 
then general Tcu dependence will be reduced to the simple sinusoidal form with period 7r/4. 
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FIG. 5: Modulation diagram in the case when the magnetic field applied in xy-plane. Solid 
separation lines correspond to the case of /3 = 7r/4 and dashed lines to the case of /? = 0. Patterned 
area corresponds to the region where the type of the solution for q and n could be changed during 
rotation along z axis. 




FIG. 6: Transition temperature dependence on the angle /3 of the magnetic field in xy-plane. For 

illustration we have chosen = e = 0, e^; = O.I7, g (normalized to J^jj^) equal to 100. There are 
switching between two types of the solution: (g > 0, n > 0) and {q = 0,n = max). 
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FIG. 7: Transition temperature dependence on the angle (3 of the magnetic field in xy-plane. For 
illustration we have chosen £z = e = 0, Ex = — O.I7, g (normalized to 7C^^) equal to 40. There are 
switching between two types of the solution: (g > 0, n > 0) and {q = max, n = 0). The inset shows 
zoom of the region near switching point. Tc line consists of several curves each corresponding to a 
different Landau level n (n = 0, 1 and 2 in this case). 

VI. CUBIC SYMMETRY 



In the case of cubic symmetry is equal to and £ — Sx — s. In the absence of the 
orbital effect the direction of modulation will be along one of the axis if e > 0, and along 
one of the main diagonals if e < 0. In the presence of orbital effect when magnetic field is 
applied along one of the cubic axis the type of solution for maximum a{H, T) depends only 
on the sign of £. If £ < then q is equal to ^J^Uq and n — [f Clr'^o — |] ■ For £ > there 
will be the choice between highest and zero modulations. If uq > 1^'^ then we obtain the 
maximum modulation along the field q — \Juq — ^Jj^ and Landau level n will be zero. But 
when Uq < 7^ff^ it is more favorable to have zero (or some residual due to the mismatch) 
modulation q and highest Landau level n = [^^jjUQ — |] • In the latter case when the 
modulation along the field is absent {q ~ 0) it turns out that the maximum Landau level 
can not be higher than 3. In the case with magnetic field applied along the one of the 
diagonals we have the same situation but with the opposite sign of e. Note that in all these 
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cases the momentum and Landau level do not depend on e at the first approximation. 



VII. DISCUSSION 



We investigated the influence of the crystal structure effects on the FFLO state based 
on the modified Ginzburg-Landau approach. We analyzed the possible solutions for the 
FFLO modulation vector and relevant Landau level functions. We have used the single- 
level approximation, but we believe that qualitatively our results would remain valid even 
if we take into account the general multi-level representation of the order parameter. For 
illustration we have restricted ourself to the tetragonal symmetry because most promising 
material for FFLO realization CeCoIn5 has namely this type of the symmetry. Our results 
can be easily generalized to any symmetry as long as deviation of the Fermi surface from 
the elliptic form can be treated as a perturbation. In the opposite case the single-level 
Landau function solution will be transformed into a series of higher level functions. Also 
this will lead to the broadening of the g = region shown in Fig. 3, which means that for a 
wide range of parameters in such a case there will be no more modulation along the field. 
The form of the Fermi surface determines the direction of the FFLO modulation in the pure 
paramagnetic limit. We see that in the presence of the orbital effect the system tries in some 
way to reproduce this optimal directions of the FFLO modulation by varying the Landau 
level index n and wave- vector of the modulation along the field. 

The higher Landau level solutions has been predicted for the FFLO phase in 2D su- 
perconductors in tilted magnetic field [5-7], in 3D rf-wavc and quasi-2D s- and ci-wave 
superconductors [11, 27], and in 3D isotropic superconductors at low temperature provided 
the Maki parameter is large [4]. Here we have demonstrated that for certain field orienta- 
tions such states naturally appear in real 3D compounds in a whole region of the FFLO 
phase existence (without any restriction to the value of Maki parameter). This behavior is 
related with crystal structure and/or pairing symmetry effects. The isotropic models used 
so far to describe FFLO state fail to predict these different types of the scenarios of the 
FFLO transitions. Indeed following the isotropic (or quasi-isotropic) model the transition 
to the FFLO state with the increase of the magnetic field always occurs via the modulation 
appearance along the field direction. On the contrary in the present paper we predict the 
FFLO transition as a formation of the higher Landau level states. The vortex state that 
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corresponds to these higher Landau level solutions have a rather complicated structure due 
to the competition between two length scales, the average distance between vortices and the 
FFLO period [12, 26, 28]. Recently in [29] the very special vortex phases with spatial hne 
nodes forming a variety of 3D spatial configurations has been predicted. Therefore we may 
expect that the mixed state in the FFLO superconductor may be very different from the 
usual Abrikosov lattice, provided that the higher Landau level solutions are realized. The 
experimentally verified consequences of these scenario of the FFLO transition are the first 
order transitions between the states with different Landau level solutions (namely between 
n — and n — 1), accompanying by the strong change of the vortex lattice structure. The 
standard experimental techniques of the vortex lattice observation (including the neutron 
scattering) could be used to detect these transformations. 

It is commonly believed that the FFLO state in CeColns corresponds to the state with 
the modulation along the magnetic field for both field orientations: along the tetragonal 
axis and in the basal plane. However, comparing the {e, Sz) diagrams (Figs. 2 and 4) we see 
that the situation when we have a zero Landau level solution for this two field orientations 
is improbable. In CeCoIns the crystal structure effects are rather important - for example 
in [30] the vortex lattice reorientation transition have been reported as well as in-plane 
anisotropy of the upper critical field [31]. In such a case we can expect that for one of 
these field orientations the Landau level solution with n > 1 may be realized. Note that 
such a possibility in connection with the FFLO state in CeCoIn5 has been discussed in 
[32]. If the crystal structure effects are large enough for the Landau level solutions with 
n > 1 the modulation along the field may be absent. Very recently [33] the modulated 
antiferromagnetic ordering has been reported in the low temperature superconducting phase 
of CeCoIns at the magnetic field in the basal plane. The antiferromagnetic ordering plays 
for the FFLO state the role of the crystal structure effect favoring the orientation of the 
FFLO modulation wavevector along the antiferromagnetic one [13]. The texture in the 
superconducting order parameter revealed by NMR experiments looks different for different 
field orientations [9] as well as the anomaly in the local magnetic inductor measurements [34] . 
This may indicate on the different types of the FFLO state for different field orientations. 
Presumably for the field orientation in the basal plane there are no FFLO modulation along 
the field. 
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